An Exploration of the Outcomes of Utilizing Ill-structured Problems in Pre=servie Teacher Preparation

The researchers of this collective case study sought to explore and understand the population of a cohort group of 22 pre-service teachers in agricultural education at the University of Illinois at Urbana-Champaign. The purpose of this study was to determine the outcomes of using problem-based learning cases by pre-service teachers to learn how to manage difficult student problems related to FFA supervision. Pre-service teachers faced difficulties in making decisions regarding ill-structured problems. They expressed challenges related to thinking of creative alternatives, considering the interests of others, and pursuing legal action if necessary in reaching solutions that were fair, relevant, appropriate, and effective. Ill-structured problems engaged pre-service teachers to be creative and reflective problem-solvers in making decisions. The pre-service teachers learned how to creatively generate alternatives, determine potential consequences for each alternative, consider possible implications of the consequences, and make decisions in a more reasonable period of time. Finally, pre-service teachers indicated that the ill-structured problems related to FFA supervision prepared them to solve similar problems that they may face as future agriculture teachers and exposed them to a variety of situations that they did not consider before the problem-based learning experience. Introduction and Theoretical Framework Beginning teachers face many illstructured, complex problems. Among their many needs, novice teachers of agriculture have problems managing and advising FFA activities (Edwards & Briers, 1999; Garton & Chung, 1996; Layfield & Dobbins, 2000; Mundt, 1991; Mundt & Connors, 1999; Talbert, Camp, & Heath-Camp, 1994) and disciplining students (Joerger & Boettcher, 2000; Mundt; Mundt & Connors; Nesbitt & Mundt, 1993; Talbert et al.). Because teacher educators play an important role in developing teachers (American Council on Education, 1999; McGhee & Cheek, 1990) and shaping the future of agricultural education (Anderson, 1977), preparing pre-service teachers through problem-based learning may help them become more reflective practitioners more capable of solving the complex, ill-structured problems they may face in their initial years of teaching. Conceptually, teacher thinking and problem-solving are not new to agricultural education. Problem-solving approaches to teaching have been a standard method in agricultural education programs (Brown, 1998), and have changed very little from the early days of agricultural education (Straquadine & Egelund, 1992). One of the basic tenets of traditional problem-solving methods is the existence of a clearly defined problem (Hedges, 1996; Newcomb, McCracken, & Warmbrod, 1993; Stewart, 1950; Sutherland, 1948). Yet, researchers and practitioners alike would agree that the profession of teaching requires educators to make decisions about and act upon problems that are often complex and ill-structured. Schön (1983) noted that teaching as a profession requires the skill of inquiring into and reflecting upon complex problems, often in very ill-structured, demanding contexts. Ball & Knobloch An Exploration of the Outcomes... Journal of Agricultural Education 63 Volume 45, Number 2, 2004 Research comparing novices to experts in their cognitive processing abilities (Sweller, 1988) reveals that experts have more well-developed schemas or mechanisms for structuring and recognizing problems. Novices, as such, must refer to more rudimentary and generic mechanisms for approaching problem situations. Furthermore, research on the practice of expert versus novice teachers indicates that expert teachers have more efficient mechanisms for thinking about and solving the problems of practice, with student development as the central outcome (Fuller, 1970). While advanced cognitive processing abilities such as critical and creative thinking, analytical thinking, higher-order thinking, and metacognition have been espoused as the aims of an educational system, research regarding the widening gap between theory and practice in professional education indicates that few pre-service teachers leave higher education with the ability to think, process, and solve the ill-structured problems of practice. The theoretical framework for the study was derived from Jonassen’s (2000) meta-theory of problem-solving. One of the basic tenets of this theory (Jonassen) is that there are critical differences between well-structured and ill-structured problems, and that ability to solve well-structured problems does not translate to ability to solve complex, ill-structured problems. As opposed to well-structured problems, ill-structured problems are authentic or emergent, have unpredictable solutions, appear ill-structured because one or more of the elements are unknown, possess multiple solutions or solution paths, possess multiple criteria for evaluating solutions, and require learners to make judgments or are dependent upon learners’ epistemological beliefs (Jonassen, 2000). The use of authentic, problem-based cases is one method of creating a learning environment more similar to the types of problems encountered in professional contexts (Jonassen, 2000). Ill-structured problems tend to be highly bound by context. As such the context of teaching and learning for teachers of agriculture is potentially shaped by student, teacher, school, and FFA program related factors. Thus, pre-service teachers of agriculture who learn how to manage, supervise, and discipline students in the role of an FFA advisor by reflecting upon ill-structured as opposed to well-defined problems would feel more prepared to handle difficult student problems in the future. The utilization of ill-structured problem-based cases has been determined to develop students’ abilities to think and process at higher cognitive levels (Hernandez-Serrano, & Jonassen, in press). Further, problembased case methods have been established as sound pedagogical tools for pre-service teacher education (Silverman, Welty, & Lyon, 2000). Yet, a paucity of research exists regarding the outcomes of utilizing illstructured problems, as a pedagogical tool for the preparation of pre-service teachers of agriculture for the problems unique to the context of their practice. Research is needed to determine the outcomes of utilizing problem-based cases in agricultural education. The findings of such research could serve as a mechanism to help developing teachers solve ill-structured and complex problems related to the practice of supervising FFA activities. Purpose and Objectives The purpose of this study was to understand the outcomes of using problembased learning cases by pre-service teachers to learn how to manage difficult student problems related to FFA supervision. The objectives of the study were to: (1) understand the perceptions of pre-service teachers regarding the problem-solving outcomes of utilizing the problem-based learning method to solve ill-structured problems; (2) identify the common themes of pre-service teachers’ learning outcomes about FFA supervision that prepared them to deal with ill-structured student problems as future teachers; and, (3) identify common themes of difficulties pre-service teachers’ faced when making decisions related to the ill-structured problems. Methods and Procedures The researchers of this collective case study (Stake, 2000) sought to explore and Ball & Knobloch An Exploration of the Outcomes... Journal of Agricultural Education 64 Volume 45, Number 2, 2004 understand the population of a cohort group of pre-service teachers in agricultural education at the University of Illinois. The target population that the researchers sought to generalize to consisted of a census of 22 pre-service teachers in a teacher education seminar. The seminar was conducted one semester before the student teaching internship. The students were randomly assigned ill-structured problems designed to prepare future agriculture teachers to solve difficult student problems related to the supervision of the FFA chapter. The illstructured problems were written by the researchers based on authentic cases and involved: (1) theft on a field trip; (2) sexual activity in a motel room; (3) drug and alcohol use on a camping trip; (4) violation of good conduct policy; (5) academic ineligibility of the FFA chapter president; and (6) horseplay on the school bus at a convention. Three to four pre-service teachers were randomly assigned the same ill-structured problems and the groups discussed how to solve the problems using the satisficing or administrative decisionmaking model (Hoy & Miskel, 2001). The satisficing decision-making model is a model for solving ill-structured problems that involves defining a problem, considering the factors related to the problem, reflecting upon all possible problem-solution outcomes, and selecting the problem-solution that is most satisfactory relative to all of the factors involved (Hoy & Miskel, 2001). In short, the satisficing model is a model of decision making for program managers, who in ill-structured contexts, recognize that multiple solution paths exist, but that the solution chosen should maximize positive outcomes while minimizing the negative outcomes of a problem. The pre-service teachers were encouraged to discuss their decision-making strategies in groups, but were asked to turn in assignments that were completed independently. The students were given three weeks to work on the assignment, which included three 50-minute sessions of in-class instruction: (a) establishing expectations; (b) making decisions using the administrative, satisficing model; and, (c) writing a ‘bad news’ letter. Students processed their ideas among assigned groups, and the class shared individual problem-solving strategies among the large group, upon completion of the individual decision-making tasks. Each pre-service teacher identified the short-term and long-term problems, analyzed the difficulties, defined a set of criteria for a satisfactory solution, outlined 17 alternatives and and explained a plan of action for their assigned ill-structured problem. This protocol including: identification of short and long term problems, analysis, defining criteria, outlining alternatives and designating a plan of action is in accordance with the satisficing decision-making model (Hoy & Miskel, 2001). Then, pre-service teachers were taught how to write a ‘bad news’ or indirect letter to a parent of one of the students involved in the ill-structured problems. The pre-service teachers were provided scaffolding and examples of how to complete the major steps of the satisficing model and ‘bad news’ letter. Students completed a self-reflection in the seminar on problem-based learning on the day that they turned in their completed assignments. All of the pre-service teachers (N = 22) completed the reflection activity. The reflection instrument used to collect the data of this study was a questionnaire containing nine items. The researchers created the instrument based on Jonassen’s (2000) meta-theory of ill-structured problems. To measure objective one, five summated rating scale items (see Table 1) were included in the questionnaire that were designed to assess pre-service teachers’ beliefs of problem-based learning. The scale was: (1) strongly disagree; (2) moderately disagree; (3) slightly disagree; (4) slightly agree; (5) moderately agree; and (6) strongly agree. In addition, four open-ended questions were used to ascertain the teachers’ thoughts regarding the use of ill-structured problems related to FFA supervision in agricultural education. Two open-ended questions on the reflection instrument addressed Objective 2: Learning Outcomes: What do you know now about FFA supervision that you didn’t know before you completed this assignment? How did this case study prepare you to deal with student problems as an agriculture Ball & Knobloch An Exploration of the Outcomes... Journal of Agricultural Education 65 Volume 45, Number 2, 2004 teacher? Finally, two open-ended questions on the reflection instrument addressed Objective 3: Difficulties: What was the most difficult aspect of trying to make a decision regarding your case? As a future agriculture teacher, how will you approach making difficult decisions related to supervising students? Students were asked not to put their names on the questionnaires to protect the anonymity of, thereby preserving the integrity of, student responses. A panel of three teacher educators in Agricultural Education at the University of Illinois established content validity. The instrument was field tested with graduate students, in agricultural education, to establish face validity. A post hoc reliability test was conducted to establish reliability. Cronbach’s (1951) alpha coefficient was 0.79 for the five, summated rating scale items (see Table 1) in the problem-solving domain. Descriptive statistics were used to analyze the numerical data for Objective 1. The data set was analyzed using SPSS. The ordinal-level data from the five summated rating scale items were reported as frequencies. These five items were summed as the problem-solving domain and was reported as a population mean and standard deviation. For Objectives 2 and 3, the researchers’ collected and interpreted the data using qualitative methods from a postpositivist epistemological stance (Lincoln & Denzin, 2000). Paper, pencils, and highlighter markers were used to help create organizers to code and summarize the qualitative data. Coding was used to analyze the qualitative data from the openended questions. The researchers created a coding scheme of the major concepts, central ideas, or related responses (Glesne, 1999). Trustworthiness and believability was established through the use of peer debriefing, member checks, an audit trail, and a reflexive journal (Donmoyer, 2001; Lincoln & Guba, 1985). Results and Findings Objective one was to understand the perceptions of pre-service teachers regarding the problem-solving outcomes of utilizing the problem-based learning method to solve ill-structured problems. The mean of the problem-solving domain was μ = 4.89 (σ = .70, N = 22). Table 1 reports the frequencies as percentages (N in parentheses) of the five items that comprised the problem-solving domain. Approximately 90% of the pre-service teachers agreed that they learned how to make decisions. Approximately 86% of the pre-service teachers agreed that they were more prepared to deal with student problems after the problem-based learning experience. Approximately 95% of the pre-service teachers agreed that the problem-based learning experience engaged them to think reflectively. Approximately 95% of the preservice teachers reported that the illstructured problems helped prepare them for similar situations they would face as agriculture teachers. All of the pre-service teachers agreed that the ill-structured problems engaged them to think of creative alternatives. Objective two was to identify the common themes of pre-service teachers’ learning outcomes about FFA supervision that prepared them to deal with ill-structured student problems as future teachers. The eight themes that subsequently emerged from coding the data for this objective are discussed in order of the most frequently mentioned to the least frequently mentioned by pre-service teachers. Decision-Making Process The first theme centered on aspects of engaging in the decision-making process. Pre-service teachers noted this outcome two times more often than any other theme. The pre-service teachers emphasized that the process of generating 17 alternatives caused them to think creatively. Pre-service teachers also stressed that they considered many factors when outlining the consequences for each alternative. They also stated that they learned to reflect upon the implications of the consequences for the students, parents, and the school community. A representative response from a pre-service teacher was, “There are many ways to think of solutions. Many will work, but some are more right than others.” Ball & Knobloch An Exploration of the Outcomes... Journal of Agricultural Education 66 Volume 45, Number 2, 2004 Table 1 Pre-service Teachers’ Perceptions of Learning Outcomes as a Result of a Problem-Based Learning Experience (N = 22) Level of Agreement, % (f) Reflection Item